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Observations  from  fleet  exercises  indicate  that  access  to  reliable  information  is  vital  to  timely,  coordinated  decision 
making.  Under  conditions  of  degraded  tactical  information,  decisions  are  often  delayed  and  alternate  sources  of  relevant  data  are 
ignored;  hence  decisions  are  often  based  on  an  incorrect  assessment  of  the  situation.  Common  ways  in  which  tactical  information 
can  be  degraded  include  (a)  periodic  loss  of  data  and/or  voice  communications,  (b)  conflicting  or  delayed  reports  from  different 
'  urces,  and  (c)  ei  roneous  data  mixed  with  veridical  data.  Empirical  results  concerning  the  specific  impact  of  these  factors  are 
needed  as  a  basis  for  modeling  teams  of  decision  makers. 

In  order  to  overcome  many  of  the  limitations  of  collecting  data  in  fleet  exercises,  the  RESA  simulation  laboratory  at  the 
Naval  Ocean  Systems  (Center  was  used  for  the  studies  reported  here.  RESA  simulations  are  large-.scale,  event-driven  battle 
simulations  in  which  v'arfare  commanders  are  presented  with  highly  realistic,  prescripted  information  streams  to  which  they 
must  respond.  RESA  provides  high  fidelity  simulation  of  platform,  weapon,  jamming,  and  other  environmental  and  hardware 
parameters. 

A  series  of  man-in-the-loop  simulation  experiments  was  conducted  in  the  RESA  facility  to  investigate  the  impact  of 
degraded  communications  on  tactical  decision  making.  These  experiments  focused  on  the  naval  antiair  warfare  (AAW) 
commander  during  an  outer  air  battle  scenario.  Naval  officers  who  were  experienced  in  battle  group  AAW  participated  in  the 
experiments,  serving  as  the  commander.  Each  officer  was  presented  with  several  battle  simulation  trials.  Tactical  information 
degradation  was  varied  in  tnese  different  trials.  The  primary  responsibilities  of  the  officers  were  to  integrate  and  disseminate 
tactical  information  from  different  sources  and  to  oversee  and  control  the  engagement  of  enemy  aircraft.  Several  types  of 
performance  measures  were  collected  during  each  simulation  run,  including  the  officers’  ongoing  verbalizations  of  their  evolving 
decision  process. 

Overall  findings  indicated  that  periodic  interruptions  in  data  and  tactical  voice  communications  degrade  battle 
effectiveness.  This  was  largely  the  result  of  delays  in  the  decision  to  launch  additional  interceptor  aircraft  as  needed  to  counter 
the  threat.  The  results  suggested  that  the  decision  delays  in  launching  interceptors  were  attributed  to  the  commander’s 
uncertainty  about  the  tactical  situation  due  to  the  interruption  in  updates  of  his  geographical  situation  display.  Few  attempts  to 
compensate  for  the  communications  losses  were  observed.  Individual  differences  were  noted  in  the  extent  to  which  the 
commanders  relied  on  various  kinds  of  tactical  information  and  in  their  tendency  to  anticipate  enemy  actions.  When  multiple 
AAW  decision  makers  (E-2  commanders)  were  dealing  with  overlapping  sectors  in  the  same  simulations,  few  attempts  to 
coordinate  their  activities  were  noted.  Rather,  each  decision  maker  functioned  independently  for  the  most  part,  as  though  leaving 
coordination  decisions  to  *heir  superior.  - 

These  findings  suggest  that  normative  modeling  efforts  should  characterize  AAW  decision  making  teams  as  tending  to 
require  confirmation  of  threats  before  reacting  with  well-defined  response  sequences.  Interruption  of  the  flow  of  tactical 
communications  is  especially  disruptive,  since  this  denies  the  commander  data  that  he  uses  to  trigger  particular  responses.  Aiding 
techniques  that  help  the  commander  to  define  and  structure  his  response  options  may  be  of  potential  value.  Such  techniques 
coulfl  help  to  minimize  the  effects  of  interrupted  communications  by  alerting  commanders  to  the  dynamic  nature  of  the  tactical 
situation. 
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Communication  interrupt  Effects  on  tactical  Decisions 


Abstract 


Observations  from  fleet  exercises  indicate  that  access  to  reliable  information  is  vital  to 
timely,  coordinated  decision  making.  Under  conditions  of  degraded  tactical  information,  decisions 
are  often  delayed  and  alternate  sources  of  relevant  data  are  ignored;  hence  decisions  are  often  based 
on  an  incorrect  assessment  of  the  situation.  Common  ways  in  which  tactical  information  can  be 
degraded  include  (a)  periodic  loss  of  data  and/or  voice  communications,  (b)  conflicting  or  delayed 
reports  from  different  sources,  and  (c)  erroneous  data  mixed  with  veridical  data.  Empirical  results 
concerning  the  specific  impact  of  these  factors  are  needed  as  a  basis  for  modeling  teams  of  decision 
makers. 

In  order  to  overcome  many  of  the  limitations  of  collecting  data  in  fleet  exercises,  the  RESA 
simulation  laboratory  at  the  Naval  Ocean  Systems  Center  was  used  for  the  studies  reported  here. 
RESA  simulations  are  large-scale,  event-driven  battle  simulations  in  which  warfare  commanders 
are  presented  with  highly  realistic,  prescripted  information  streams  to  which  they  must  respond. 
RESA  provides  high  fidelity  simulation  of  platform,  weapon,  jamming,  and  other  environmental 
and  hardware  parameters. 

A  series  of  man-in-the-loop  simulation  experiments  was  conducted  in  the  RESA  facility  to 
investigate  the  impact  of  degraded  communications  on  tactical  decision  making.  These  experiments 
focused  on  the  naval  anti-air  warfare  (AAW)  commander  during  an  outer  air  battle  scenario.  Naval 
officers  who  were  experienced  in  battle  group  AAW  participate  in  the  experiments,  serving  as  the 
commander.  Each  officer  was  presented  with  several  battle  simulation  trials.  Tactical  information 
degradation  was  varied  in  these  different  trials.  The  primary  responsibilities  of  the  officers  were  to 
integrate  and  disseminate  tactical  information  from  different  sources  and  to  oversee  and  control  the 
engagement  of  enemy  aircraft.  Several  types  of  performance  measures  were  collected  during  each 
simulation  run,  including  the  officers’  ongoing  verbalizations  of  their  evolving  decision  process. 

Overall  findings  indicated  that  periodic  interruptions  in  data  and  tactical  voice 
communications  degrade  battle  effectiveness.  This  was  largely  the  result  of  delays  in  the  decision 
to  launch  additional  interceptor  aircraft  as  needed  to  counter  the  threat.  The  results  suggested  that 
the  decision  delays  in  launching  interceptors  were  attributed  to  the  commander’s  uncertainty  about 
the  tactical  situation  due  to  the  interruption  in  updates  of  his  geographic  situation  display.  Few 
attempts  to  compensate  for  the  communications  losses  were  observed.  Individual  differences  were 
noted  in  the  extent  to  which  the  commanders  relied  on  various  kinds  of  tactical  information  and  in 
their  tendency  to  anticipate  enemy  actions.  When  multiple  AAW  decision  makers  (E-2 
commanders)  were  dealing  with  overlapping  sectors  in  the  same  simulations,  few  attempts  to 
coordinate  their  activities  were  noted.  Rather,  each  decision  maker  functioned  independently  for 
the  most  part,  as  though  leaving  coordination  decisions  to  their  superior. 

These  findings  suggest  that  normative  modeling  efforts  should  characterize  AAW  decision 
making  teams  as  tending  to  require  confirmation  of  threats  before  reacting  with  well-defined  .jT 
response  sequences.  Interruption  of  the  flow  of  tactical  communications  is  especially  disruptive,  - 
since  this  denies  the  commander  data  that  he  uses  to  trigger  particular  responses.  Aiding 
techniques  that  help  the  commander  to  define  and  structure  his  response  options  may  be  of 
potential  value.  Such  techniques  could  help  to  minimize  the  effects  of  interrupted  communications 
by  alerting  commanders  to  the  dynamic  nature  of  the  tactical  situation.  ^ 
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Command  performance  and  system  effectiveness  have  been  measured  during  a  series  of 
experiments  examining  the  ability  of  naval  battle  force  commanders  to  monitor  and  control  air 
defense  activities  during  information  degradation  (delay,  conflict,  and  interruption).  The  research 
program  uses  these  measures,  along  with  results  of  fleet  exercises  and  small-scale  experiments,  to 
develop  improved  guidelines  for  naval  and  weapons  systems  and  to  evaluate  decision  aids  and 
decision  support  tools.  Specific  objectives  of  these  experiments  were  to: 

•  Examine  the  effect  of  degraded  information  states  on  a  critical  decision  in  air  defense. 

•  Describe  how  information  use  changes  as  the  air  defense  scenario  evolves. 

•  Explore  individual  differences  in  tactical  information  use  and  decision  making. 

•  Examine  the  cognitive  structures  associated  with  tactical  decision  making. 

Introduction 

One  of  the  first  steps  in  establishing  the  architecture  of  a  navy  system  and  developing  its 
design  guidelines  is  to  understand  the  communication  needs  of  the  commander.  Unfortunately, 
command  decision  making  is  not  well  understood,  and  commanders’  requirements  for  information 
are  not  easily  specified.  The  description  of  the  tactical  situation  available  to  a  commander  is 
incomplete,  evolves  with  variable  time  delays,  and  contains  noise  and  false  signals.  The 
consequences  of  noise,  error,  delay,  and  information  interruption  on  combat  decision  processes 
have  not  been  determined,  because  information  use  by  humans  in  these  complex  environments  has 
not  been  systematically  measured  or  analyzed.  In  order  to  determine  how  commanders  monitor, 
classify,  and  integrate  the  large  amounts  of  information  in  systems,  the  following  cognitive 
factors  need  to  be  examined:  (a)  the  types  of  information  used  by  the  commander,  (b)  the 
frequency  of  use  of  that  information,  and  (c)  how  the  pattern  of  information  use  changes  with  the 
tactical  situation.  Documentation  of  the  cognitive  proces.ses  of  the  decision  maker  in  battle  is 
prerequisite  to  designing  support  tools  and  aids  that  can  enhance  combat  system  performance  and 


improve  battle  outcome.  Specifications  for  combat  systems  must  be  based  on  as  complete  a 
description  of  these  cognitive  processes  as  possible. 


This  paper  summarizes  the  results  of  the  first  four  simulation  experiments  of  a  series  that 
was  begun  in  1988  in  the  RESA  laboratory  at  the  Naval  Ocean  Systems  Center,  San  Diego.  The 
RESA  laboratory  presents  large-scale,  event-driven  battle  simulations  with  realistic  scenarios. 
High  fidelity  simulation  is  available  for  multiple  platforms,  sensors,  and  weapons  under  a  complete 
range  of  tactical  and  environmental  situations.  The  RESA  laboratory  has  been  used  primarily  for 
training  naval  battle  force  staffs  and  for  studies  of  advanced  sensor  and  weapons  concepts.  Our 
interest  was  to  use  the  RESA  facility  for  man-in-the-loop  studies  to  determine  the  cognitive  and 
decision  processes  of  warfare  commanders  during  battle. 

Initial  experiments  in  this  series  of  simulations  have  dealt  with  one  or  two  commanders 
responsible  for  the  air  defense  of  the  battle  force  against  a  large  raid  of  enemy  aircraft.  The 
experiments  measured  the  overall  effectiveness  of  the  system  with  the  performance  of  the 
commanders  as  its  principal  determining  factor. 

Technical  Approach 

The  tactical  scenario  for  the  experiments  has  been  the  air  defense  of  a  two  carrier  battle 
force  against  a  large  raid  of  enemy  bombers  carrying  air-to-surface  missiles.  The  battle  force 
included  eight  Aegis  cruisers,  four  destroyers,  and  a  fast  combat  support  ship.  The  principal 
weapon  employed  was  the  F14D  fighter-interceptor  armed  with  modem  air-to-air  missiles.  A  hot 
war  situation  was  assumed  throughout.  The  simulation  runs  consisted  of  two  hours  of  operational 
time  compressed  into  50  minutes  of  real  time.  Detailed  intelligence  and  instructions  were  provided 
to  all  participants,  and  research  assistants  interacted  with  the  computer  system  so  that  the 
commander  was  not  required  to  learn  simulation  language  or  procedures.  In  order  to  standardize 
the  scenario  for  the  purpose  of  comparing  the  performance  of  different  commanders,  enemy 
actions  were  entirely  preprogrammed  for  all  simulation  runs. 


The  RESA  facility  contains  several  command  centers,  each  of  which  can  be  programmed  to 
view  events  from  the  perspective  of  a  major  organizational  component  of  the  battle  force. 
Command  center  work  stations  were  set  up  to  simulate  standard  navy  tactical  displays.  Each 
consisted  of  a  geographic  display  (GEO-Plot)  that  presented  game  objects  in  standard  navy  tactical 
symbology  and  automatic  status  boards  that  presented  various  alphanumeric  data,  including 
weapon,  track  and  sensor  data. 

Participants 

Each  experiment  employed  between  12  and  32  experienced,  active  duty  naval  officer 
volunteers  from  the  San  Diego  area  for  a  half-day  session,  consisting  of  three  to  five  simulation 
runs. 

Data  Collection 

The  RESA  software  automatically  recorded  engagement,  orders  and  position  information 
each  minute.  These  data  were  coded  and  stored  on  magnetic  tape  and  disks  at  the  conclusion  of 
each  run.  In  addition,  participants  made  verbal  reports  and  were  encouraged  to  think  aloud  in 
order  to  reveal  cognitive  factors  influencing  their  decisions.  Their  comments  and  verbal 
communications  were  recorded  on  audio  tape  for  subsequent  protocol  analysis.  All  relevant 
measures  of  effectiveness  and  performance  were  recorded  and  analyzed,  including: 


Interceptor  launch  decisions. 

Communications  and  information  use. 

Individual  characteristics  of  experience  and  training. 


The  first  experiment  required  participants  to  make  a  single  decision  to  launch  all  of  their 
interceptors,  once  they  felt  they  had  determined  the  location  of  the  incoming  raid.  The  timing  of 
their  launch  decision  and  the  intercept  vector  assigned  to  the  F14D's  were  the  performance 
measures  of  major  concern.  Complete  and  accurate  tactical  information  was  available  to  the 
participants  throughout  the  entire  experiment  on  simulated  data  links. 

Experiment  2 

A  more  complex  scenario  with  increased  decision  options  was  employed  in  the  second  and 
following  experiments.  The  commander  was  given  a  range  of  tactical  options,  including  the 
opportunity  to  launch  fighters  and  to  control  them  at  will.  A  tactical  communication  degradation 
condition  was  introduced  in  which  the  tactical  data  link  and  voice  communications  circuits  among 
the  commander's  ships  and  aircraft  were  interrupted  at  prescribed  intervals.  These  interruptions 
were  designed  to  simulate  communications  link  breakdown  due  to  extensive  jamming,  equipment 
failure,  unusual  atmospheric  conditions,  or  superimposed  requirements  for  electronic  silence. 
During  periods  of  full  communications,  tactical  voice  and  digital  data  were  continuously  (100%) 
available  to  the  participants.  During  periods  of  degraded  communications,  tactical  data  were 
interrupted  50%  of  the  time.  Also,  a  parallel,  record  communication  network  was  introduced  with 
variable  time  delays  of  3-5  minutes.  Other  scenario  features  remained  constant  to  facilitate 
comparison  with  the  earlier  experiments. 

Experiment  3 

The  third  experiment  employed  a  greater  range  of  tactical  communication  interruption 
conditions.  One  third  of  the  trials  were  conducted  under  entirely  continuous  (100%) 
communications  conditions;  one  third  of  the  trials  used  a  33%  interruption  schedule;  and  one  third 
of  the  trials  had  a  67%  interrupt  schedule.  In  addition,  false  tactical  information  was  introduced  to 
increase  the  participants’  uncertainty  regarding  the  location  of  the  main  portion  of  the  raid.  The 
remaining  scenario  features  were  the  same  as  used  in  Experiments  1  and  2. 


Experiment  4 


The  fourth  experiment  examined  two  air  defense  commanders  as  they  jointly  defended  the 
battle  force  against  an  attack  of  enemy  bombers.  The  participants  were  experienced  early  warning 
and  tactical  air  conp-ol  officers  (E2C  aircraft)  from  the  Naval  Air  Station,  Miramar,  CA.  Tactical 
data  and  voice  communications  were  available  full-time  (100%)  in  half  of  the  simulations  and  part- 
time  (50%)  in  the  remainder.  In  addition,  two  tactical  situations  were  presented.  In  the  simpler, 
balanced  raid  situation,  enemy  force  arrival  was  uniformly  distributed  across  the  defense  sectors. 
In  the  more  complex,  unbalanced  raid  situation,  the  larger  portion  (60%)  of  the  raid  arrived  within 
one  E2Cs  sector  of  responsibility,  and  the  smaller  (40%)  portion  of  the  raidwithin  the  other  E2C’s 
sector.  Besides  increasing  the  participants’  uncertainty  about  the  tactical  situation,  the  complex, 
unbalanced  situation  created  an  opportunity  for  each  participant  to  exchange  information  and  to 
share  assets  to  optimize  their  defense.  Two  forms  of  communication  interruption  were  presented: 
(1)  between  the  two  E2C  commanders  and  (2)  between  each  commander  and  their  common 
superior  (AW),  played  by  a  researcher. 


Results 


Interceptor  Launch  Decisions 

The  decision  to  launch  interceptors  to  engage  the  incoming  Orange  bombers  is  one  of  the 
most  significant  command  actions  in  response  to  the  tactical  situation  in  these  scenarios.  It  is  a 
complex  decision  that  involves  multiple  launches  at  different  times,  as  determined  by  the 
commander’s  interpretation  of  the  tactical  situation.  The  decision  to  launch  interceptors  is  a  critical 
determinant  of  the  range  at  which  threats  can  be  effectively  engaged  and  the  number  of  enemy 
aircraft  that  can  be  destroyed.  Therefore,  the  time  and  rate  of  interceptor  launches  are  two  of  the 
measures  of  greatest  concern  to  this  research  program. 

Experirngnt  1 


In  the  tirst  experiment,  participants  typically  delayed  their  launch  decision  beyond  tiie 
optimum  time  needed  to  make  the  most  effective  and  complete  intercepts.  These  initial 
observations  raised  questions  about  the  organizational  and  information  processing  factors  that 
contribute  to  the  delay  and  what  difference  it  makes  to  combat  system  effectiveness.  Experiments 
2  through  4  verified  these  basic  findings  and  also  measured  the  effects  of  communications 
degradation  on  air  battle  outcome. 

Experiments  2  and  3 

The  tactical  communications  interruptions  of  Experiments  2  and  3  produced  measurable 
reductions  in  launch  rates  (Figure  1).  Compared  to  full  communications,  interrupted  tactical  data  flow 
resulted  in  significantly  delayed  launches  during  the  initial  30  minutes.  This  effect  was  especially 
pronounced  in  the  67%  and  33%  conditions.  These  launch  delays  were  correlated  with  other  measures 
of  effectiveness  and  performance  [1,2].  In  addition,  not  all  aircraft  were  launched,  regardless  of 
communications  condition,  partially  due  to  inadequate  time  remaining  to  intercept  the  raid  (shown  in 
the  Figure  at  55  minutes).  The  tactical  information  time  delay  of  Experiment  2  and  the  use  of  false 

tactical  information  in  Experiment  3  did  not  significantly  impact  performance  under  these  conditions. 
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Figure  1.  Cumulative  interceptor  launches.  Experiments  2  and  3. 


Experiment  4 


Figures  2  through  4  show  averaged,  cumulative  interceptor  launches  over  the  course  of  the 
battle  simulation  in  Experiment  4.  Figure  2  shows  the  simple  tactic  (balanced  enemy  force 
distribution)  with  full  communications;  Figure  3  shows  the  complex  tactic  (unbalanced  enemy  force 
distribution)  with  communications  interruption  between  participants  (E2C-E2C);  and  Figure  4  shows 
the  complex  tactic  with  communications  interruption  between  the  participants  and  their  common 
superior  (E2-AW).  Each  figure  has  been  labeled  to  indicate  the  latest  possible  time  of  launch  to 
successfully  intercept  the  incoming  enemy  force. 
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Figure  2.  Cumulative  interceptor  launches:  Simple  tactic,  no  communications  interruption. 


Figure  3.  Cumulative  interceptor  launches:  C  jmplex  tactic,  E2C-E2C  communications  interruption. 
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Figure  4.  Cumulative  interceptor  launches:  Complex  tactic,  E2-AW  communications  interruption. 

Note  the  marked  reduction  in  percentage  of  interceptors  launched  by  time  60  from  the  simple,  tactic, 
full  communications  condition  (Mean  =  83%),  to  the  complex  tactic,  E2C-E2C  interruption  condition 
(Mean  =  67%),  to  the  complex  tactic,  E2C-AW  communications  interruption  condition  (Mean  =  62%). 
The  evidence  indicates  that  the  effect  of  information  degradation  is  compounded  by  the  complexity  and 
uncertainty  of  the  tactical  situation. 


Communications  and  Infoimation  Use 


To  analyze  the  commander's  information  requirements  and  cognitive  processes,  extensive 
\erbal  protocol  analyses  were  conducted  on  audio  tape  recordings  of  the  commander's  comments  and 
communications  in  all  of  the  experiments.  An  example  of  the  findings  is  shown  in  Figure  5. 
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Figure  5.  Information  use  profile  during  the  air  defense  battle. 


This  figure  .shows  the  commanders'  reliance  on  ESM  information  during  the  °;arly  stages  of  the 
battle,  as  well  as  the  decreasing  interest  in  location  information  (track  location)  and  the  increasing 
interest  in  raid  count  during  the  later  stages  of  the  battle.  Other  results  of  the  protocol  analyses 
describe  changes  in  communication  rates  (messages  per  minute)  during  the  course  of  battle. 

Cognitive  Network  Analysis 

A  related  pan  of  this  research  program  used  network  scaling  techniques  to  investigate 
commanders’  cognitive  structures.  Previous  research  has  shown  that  cognitive  structures  exert  a 
significant  infli'ence  on  decision  making  performance,  and  that  experts  and  novices  often  differ 


noticeably  in  cognitive  structure  [3].  In  the  present  context,  cognitive  networks  can  contribute  to 
an  improved  understanding  of  how  tactical  informatin  is  organized  in  the  decisison  maker’s  mind. 

The  cognitive  structures  reported  here  are  networks  in  which  individual  infonnation 
elements  are  depicted  as  nodes,  and  the  relationships  among  those  elements  are  depicted  as  links 
connecting  the  nodes.  A  link  is  placed  between  two  nodes  if  the  resulting  distance  forms  the 
shortest  possible  path  between  them.  Link  length  is  determined  by  weights  that  reflect  the  strength 
of  the  relationship  between  the  elements.  These  weights,  obtained  through  participants’  ratings  of 
element  similarity,  provide  a  direct  measure  of  psychological  distance  between  elements.  Elements 
rated  as  highly  similar  are  assigned  a  correspondingly  small  psychological  distance  value,  and  vice 
versa.  Networks  of  tactical  information  elements  based  on  psychological  distance  data  can  convey 
the  cognitive  organization  of  information  which  is  used  in  tactical  decision  making. 

A  rating  task  was  employed  to  collect  similarity  values  among  a  set  of  ten  tactical 
information  elements  that  were  closely  related  to  air  defense  activities.  Participants'  similarity 
judgments  were  obtained  for  all  pair-wise  combinations  of  these  elements.  The  similarities  rating 
task  was  performed  by  each  participant  both  before  and  after  the  air  defense  simulation  exercises 
described  above  as  part  of  Experiments  3  and  4. 

Empirically-Derived  Cognitive  Networks 

The  cognitive  network  based  on  the  averaged  similarity  ratings  for  all  participants  in 
Experiment  3  |2)  is  shown  in  Figure  6.  Target  Identification  is  the  most  central  element  in  the 
network,  being  linked  to  more  elements  (5)  than  any  other  element.  Note  that  link  lengths,  which 
reflect  degree  of  element  relatedness,  do  not  differ  greatly  from  each  other.  In  part,  this  is  a 
consequence  of  data  averaging;  however,  it  also  indicates  that  the  concepts  linked  directly  to  each 
other  were  judged  as  roughly  similar  in  importance.  Elements  judged  less  similar  to  each  other  arc 
separated  by  more  than  one  link.  Five  intervening  links  separate  the  two  pairs  of  elements  that  are 


least  related  to  each  other.  Interceptor  Launch  -  Emissions  Control  and  Fuel/Weapons 
Replenishment  -  Emissions  Control. 


Figure  6.  Information  element  network  for  Experiment  3. 

The  corresponding  network  obtained  from  Experiment  4  participants  [4]  is  displayed  in 
Figure  7.  Once  again,  Target  Identification  is  the  most  central  element  in  the  network,  and  is 
linked  directly  to  the  same  five  elements  as  in  the  Experiment  3  network.  As  before,  note  that  the 
link  lengths  do  not  differ  greatly  from  each  other.  Overall  then,  networks  in  Experiments  3  and  4 
are  remarkably  similar  in  configuration  and  link  length,  in  spite  of  the  fact  that  the  participants  in 
the  two  experiments  possessed  different  operational  backgrounds  and  levels  of  experience. 


Figure  7.  Information  element  network  for  Experiment  4. 


Table  1  shows  the  results  of  a  quantitative  analysis  of  the  degree  of  similarity  of  these 
networks.  The  Centrality  of  each  network  can  be  is  represented  as  the  median  of  the  distances 
from  the  t-ctical  information  elements  to  the  central  element  (Target  Identification).  The  rank  order 
correlation  between  the  Centrality  distances  of  the  two  networks  is  significant  (Kendall's  tau  = 
.592,  p  <.05),  supporting  a  conclusion  that  the  two  groups  of  officers  possess  highly  similar 
cognitive  structures  for  these  information  elements. 

Table  1 

Comparison  of  Information  Networks;  Experiments  3  and  4 


Tactical 

Information 

Element 

Difference 

Between 

Expts 

Distance  to 

Central  Element 

Expt  3  Expt  4 

Target  Identification 

0 

0 

0 

Track  Localization 

0.4 

2.1 

1.7 

Electronic  Signal  Analysis 

0.1 

1.6 

1.5 

Intelligence  Correlation 

0.3 

1.9 

1.6 

Detection  Avoidance 

0.4 

4.7 

4.3 

Rules  of  Engagement 

0.5 

1.8 

2.3 

Emission  Control 

3.1 

6.0 

2.9 

CAP  Assignment 

0.3 

2.0 

1.7 

Intercept  Launch 

1.2 

4.3 

3.1 

Fuel/Wpn  Replenishment 

0.5 

3.8 

3.3 

Median  0.4  2.1  2.3 


The  cognitive  network  analyses  will  be  continued  in  future  research  to  aid  in  understanding 
the  effects  of  information  organization  for  battle  force  commanders  and  their  staffs.  As  part  of  this 
effort,  we  will  also  examine  officers  with  a  wider  range  of  battle  group  experience  and  include 
control  groups  of  individuals  with  no  naval  warfare  training  or  experience. 

Discussion 

The  purpose  of  this  series  of  man-in-the-loop  studies  is  to  analyze  the  organizational 
structure  and  information  requirements  of  the  battle  force.  Reliable  estimates  of  information 
requirements  in  combat  can  form  a  solid  basis  for  system  design  guidelines.  The  types  and 
patterns  of  communications  among  key  personnel  and  tactical  information  flow  to  the  battle 
commander  were  focused  on  in  these  studies.  To  achieve  this  we  have  assessed  information  use 
and  decision  making  by  personnel  in  command  positions  and  relating  it  to  the  effectiveness  of  the 
total  combat  system.  Experienced  naval  officers  were  examined  under  various  tactical,  information 
loading,  and  information  degradation  conditions  during  repeated  simulation  exercises  of  a 
prolonged  air  battle.  Results  showed  that  interruption  of  tactical  communications  introduced 
significant  delays  in  critical  interceptor  launch  decisions  and  reduced  the  effectiveness  of  tactical  air 
control. 


Most  officers  were  unaware  of  the  extent  of  the  system  performance  deficit  caused  by  the 
information  degradation  and,  as  a  result,  failed  to  compensate  for  it.  They  consistently  delayed 
critical  decisions  (e.g.,  interceptor  launches)  until  the  threat  was  visible  on  radar.  This 
conservative  behavior  persisted  even  after  the  officers  had  performed  multiple  simulation  runs  with 
similar  outcomes.  Furthermore,  this  conservatism  was  exacerbated  by  tactical  situation 
complexity.  Most  participants  concentrated  their  attention  on  a  single  geographical  display, 
regardless  of  its  currency,  and  neglected  information  from  other  sources.  Most  did  not  obtain 


maximum  benefit  from  multiple  sensors  and  tended  to  disregard  contradictory  information.  Shifts 
in  the  use  of  tactical  information  were  noted  as  a  function  of  the  stage  of  battle — from  long  range 
detection  to  final,  close-in  encounters.  An  individual-difference  classification  system  was 
proposed  along  two  dimensions:  (1)  extent  of  information  used  and  (2)  conservatism  in  response 
to  tactical  information. 

Quantitative  scaling  techniques  show  promise  for  improving  understanding  of  the  cognitive 
organization  basis  of  command  decision  making.  Cognitive  network  structures,  through  their  link 
patterns  and  link  lengths,  can  explicitly  depict  the  inter-relationship  of  tactical  information  elements 
and  concepts.  The  role  of  experience  in  determining  the  organization  of  cognitive  structure  and 
information  processing  characteristics  relevant  to  air  defense  merits  further  attention. 

Conclusions  and  Recommendations 

This  experimental  series  represents  an  effort  to  gather  empirical  data  about  command 
decision  making  empirically,  in  a  realistic,  complex  environment.  Traditionally,  system  measures 
have  been  used  to  examine  the  outcome  of  warfare  simulations  without  the  measured  contribution 
of  the  decision  makers.  Although  this  experiment  demonstrated  some  notable  and  consistent 
effects  of  communications  degradation,  it  also  suggested  several  areas  to  be  explored  in  future 
research.  These  areas,  which  directly  extend  the  findings  of  this  experiment,  include: 

•  Design  and/or  test  of  decision  aids  for  the  anti-air  warfare  commander.  Decisions  such  as 
those  involved  in  interceptor  launches  can  be  aided  by  appropriate  decision  support  tools. 
Existing  aids  can  be  evaluated  and  new  concepts  developed  and  tested  in  the  RESA  simulation 
with  the  commander  in  the  loop. 


Expand  investigations  of  the  role  of  the  decision  maker  to  include  planning  functions.  The 
influence  of  preplanning  for  air  defense  is  of  recognized  importance,  particularly  to  prepare  for 
periods  of  communications  interruption  and/or  unexpected  tactical  situations. 


•  Examine  various  distributions  of  duties  and  command  structures.  Each  command  organization 
calls  for  an  appropriate  communication  architecture.  The  resulting  distributions  of  function  and 
their  communications  networks  are  likely  to  have  differing  force  multiplier  effects.  The  extent 
of  the  effects  may  be  assessed  through  continued  man-in- the-loop  simulation  research  as 
represented  by  these  experiments. 

•  Examine  the  impact  of  off-board  information  on  effectiveness.  The  influence  of  such  non- 
organic  information  depends  partly  on  how  well  it  can  be  transmitted  through  the  command 
organization  to  reach  the  proper  commanders  in  a  timely  fashion. 

•  Determine  whether  these  effects  can  be  generalized  to  other  warfare  areas.  The  findings 
concerning  command  decision  making  are  intended  to  reveal  potential  improvements  that  can  be 
used  to  enhance  systems  in  all  warfare  areas. 

•  The  cognitive  network  structures  of  tactical  decision  makers  should  be  more  thoroughly 
studied.  Determining  how  they  differ  as  a  function  of  experience  would  be  especially  valuable. 
Such  information  could  be  used  in  developing  training  programs  that  encourage  the 
development  of  cognitive  structures  associated  with  expert  decision  making  performance. 

References 

[1]  Callan,  J.  R.,  Kelly,  R.  T.,  Gwynne  III,  J.  W.,  Karstein,  R.  A.,  &  Feh6r,  B.  Patterns  of 
information  use  in  outer-air  battle  decision  making,  Experiment!.  San  Diego,  CA;  Pacific 
Science  &  Engineering  Group,  1989. 

[2]  Gwynne,  J.  W.,  Callan,  J.  R.,  Kelly,  R.  T.,  &  Feh6r,  B.  (1989).  Patterns  of  Information 
Use  in  Outer-Air  Battle  Decision  Making:  Experiment  3.  San  Diego,  CA;  Pacific  Science 
&  Engineering  Group, . 


[3]  Schvaneveldt,  R.  W.,  Durso,  F.  T.,  Goldsmith,  T.  E.,  Breen,  T.  J.,  Cooke,  N.  M., 
Tucker,  R.  G.,  &  DeMaio,  J.  C.  (1985).  Measuring  the  structure  of  expertise. 
International  Journal  of  Man-Machine  Studies,  23,  699-728. 

[4]  Callan,  J.  R.,  Gwynne  III,  J.  W.,  Pinney,  T.,  &  Feher,  B.  (1990).  Patterns  of 
information  use  in  outer-air  battle  decision  making:  Experiment  4.  San  Diego,  CA:  Pacific 
Science  &  Engineering  Group,  Inc.  (Confidential) 


